Overview

We manually curated 500\(+\) multiple modality PPMI MRI subjects. 5 messages from this effort.

Raw data quality is overall high

visual inspection by humans

Processing improves quality/usability further

failure rates humans vs processed
we provide scientists handles on this so that can be “strict” or “loose” in data selection
we document these parameters

Detailed curation results provide a unique resource in itself

for algorithm developers
- we ourselves benefit from this (will show examples)
for scientists
- to have confidence (or at least context regarding quality) in looking at automated IDPs derived from these data

Enables investigation of the “meaning” of M3RI at the individual level
Empowers investigation of the “meaning” of M3RI at the population level

the PPMI500 cohort

Reviewing multiple modality data quality for current phase of collection
we collaborated on cohort definition
- first phase - select subjects based on sampling from sites
- second phase - select subjects based on population characteristics (MJFF/PPMI)
show table of summary cohort

Table 1. Baseline cohort for subjects with T1w IDPs and non-missing diagnosis.
	CN	GenAR	LRRK2PD	SporadicAR	SporadicPD	p
	(N=47)	(N=1)	(N=4)	(N=230)	(N=217)	p
age	63.5 ± 12.5	71.9 ± NA	59.3 ± 7.1	67.7 ± 5.7	64.5 ± 9.4	< 0.001
Sex						0.026
Female	23 (48.9%)	1 (100.0%)	3 (75.0%)	102 (44.3%)	72 (33.2%)
Male	24 (51.1%)	0 ( 0.0%)	1 (25.0%)	128 (55.7%)	145 (66.8%)
duration.yrs			2.1 ± 0.7		1.0 ± 0.7	0.002
updrs.totscore	5.3 ± 3.8	14.0 ± NA	30.3 ± 13.1	12.1 ± 9.3	36.5 ± 15.0	< 0.001
CSFSAA						< 0.001
Negative	36 (85.7%)	1 (100.0%)	2 (50.0%)	82 (38.7%)	10 ( 4.9%)
Positive	6 (14.3%)	0 ( 0.0%)	2 (50.0%)	130 (61.3%)	194 (95.1%)
LEDD	0.0 ± 0.0	0.0 ± NA	259.0 ± 317.7	1.3 ± 19.8	3.6 ± 30.3	< 0.001
hasDTI						0.117
FALSE	9 (19.1%)	1 (100.0%)	1 (25.0%)	57 (24.8%)	38 (17.5%)
TRUE	38 (80.9%)	0 ( 0.0%)	3 (75.0%)	173 (75.2%)	179 (82.5%)
hasfMRI						0.003
FALSE	5 (10.6%)	1 (100.0%)	0 ( 0.0%)	12 ( 5.2%)	15 ( 6.9%)
TRUE	42 (89.4%)	0 ( 0.0%)	4 (100.0%)	218 (94.8%)	202 (93.1%)

1. Data quality is overall high

How do we define M3RI quality?

we manually QC’d each modality for each of 500 baseline subjects
- characterize reasons for failure

what is the QC protocol?

Review QC protocol – generally speaking

what are the QC results?

Summarize QC results for each modality
- raw data QC and failure rates
Summarize results for “joint” quality - how many subjects are high quality across all modalities at once?

Tables of Manual QC results

Table 2 Patterns of QC results by modality: FLAIR
	T2Flair	p
	(N=549)	p
qcfail_orientation
True	1 (100.0%)
qcfail_intensity_saturation
True	4 (100.0%)
qcfail_background_noise
True	6 (100.0%)
qcfail_motion
True	10 (100.0%)
qcfail_other
True	2 (100.0%)
qchuman_FLAIR		< 0.001
False	25 ( 4.6%)
True	524 (95.4%)
has_humanqc
1	549 (100.0%)

Table 3 Patterns of QC results by modality: T1w
	T1w	p
	(N=523)	p
qcfail_background_noise
True	2 (100.0%)
qcfail_motion
True	4 (100.0%)
qcfail_other
True	4 (100.0%)
qchuman_T1w		< 0.001
False	10 ( 1.9%)
True	513 (98.1%)
has_humanqc
1	523 (100.0%)

Table 4 Patterns of QC results by modality: NM
	Neuromelanin	p
	(N=502)	p
NM_QC_Ratings_Z		< 0.001
0	88 (18.3%)
1	393 (81.7%)
NM_QC_Ratings_SNVol		< 0.001
0	7 ( 1.5%)
1	474 (98.5%)
NM_QC_Ratings_AvgIntensity		< 0.001
0	5 ( 1.0%)
1	473 (99.0%)
NM_QC_Ratings_SDIntensity		< 0.001
0	5 ( 1.0%)
1	473 (99.0%)
NM_QC_Ratings_MaxIntensity		< 0.001
0	5 ( 1.0%)
1	473 (99.0%)
NM_QC_Ratings_Failures		< 0.001
0	386 (80.8%)
1	81 (16.9%)
2	6 ( 1.3%)
3	4 ( 0.8%)
4	1 ( 0.2%)

Table 5 Patterns of QC results by modality: DTI
	DTIb0	DTIdwi	p
	(N=1204)	(N=1340)	p
qcfail_orientation
True	16 (100.0%)	16 (100.0%)
qcfail_phantom
True	6 (100.0%)	6 (100.0%)
qcfail_intensity_saturation
True	0 ( 0.0%)	4 (100.0%)
qcfail_background_noise
True	50 (100.0%)	54 (100.0%)
qcfail_motion
True	54 (100.0%)	70 (100.0%)
qcfail_signal_dropout
True	14 (100.0%)	22 (100.0%)
qcfail_other			1.000
GHOSTING	10 (62.5%)	10 (62.5%)
True	6 (37.5%)	6 (37.5%)
qchuman_DTI			0.481
False	94 ( 7.8%)	116 ( 8.7%)
True	1110 (92.2%)	1224 (91.3%)
has_humanqc
1	1204 (100.0%)	1340 (100.0%)

Table 6 Patterns of QC results by modality: rsfMRI
	rsfMRI	rsfMRI_LR	rsfMRI_RL	p
	(N=2969)	(N=187)	(N=854)	p
qcfail_orientation
True	9 (100.0%)	1 (100.0%)	16 (100.0%)
qcfail_phantom
True	14 (100.0%)	0 ( 0.0%)	0 ( 0.0%)
qcfail_spacing
True	14 (100.0%)	0 ( 0.0%)	0 ( 0.0%)
qcfail_intensity_saturation
True	20 (100.0%)	10 (100.0%)	32 (100.0%)
qcfail_motion
True	581 (100.0%)	0 ( 0.0%)	0 ( 0.0%)
qcfail_signal_dropout
True	165 (100.0%)	15 (100.0%)	48 (100.0%)
qcfail_temporal_noise
True	1 (100.0%)	1 (100.0%)	8 (100.0%)
qcfail_insufficient_volumes
True	12 (100.0%)	0 ( 0.0%)	0 ( 0.0%)
qcfail_FOV
True	1 (100.0%)	0 ( 0.0%)	0 ( 0.0%)
qcfail_other				< 0.001
472 volumes, restart appended to initial scan?	0 ( 0.0%)	0 ( 0.0%)	8 (33.3%)
axial slices look wrong way around?	0 ( 0.0%)	1 (50.0%)	8 (33.3%)
strange artifacts in tsnr	0 ( 0.0%)	1 (50.0%)	8 (33.3%)
True	1 (50.0%)	0 ( 0.0%)	0 ( 0.0%)
wrap-around artifact	1 (50.0%)	0 ( 0.0%)	0 ( 0.0%)
qchuman_rsfMRI				0.002
False	457 (15.4%)	19 (10.2%)	96 (11.2%)
True	2512 (84.6%)	168 (89.8%)	758 (88.8%)
has_humanqc
1	2969 (100.0%)	187 (100.0%)	854 (100.0%)

T1w results

T1w QC summary
modality	Total	Successes	Failures	Percent_Success	Percent_Failure	failcat
T1w	523	513	10	98.08795	1.9120459	qchuman_T1w
T1w	523	521	2	99.61759	0.3824092	qcfail_background_noise
T1w	523	519	4	99.23518	0.7648184	qcfail_motion
T1w	523	519	4	99.23518	0.7648184	qcfail_other

Flair results

FLAIR QC summary
modality	Total	Successes	Failures	Percent_Success	Percent_Failure	failcat
T2Flair	549	524	25	95.44627	4.5537341	qchuman_FLAIR
T2Flair	549	548	1	99.81785	0.1821494	qcfail_orientation
T2Flair	549	545	4	99.27140	0.7285974	qcfail_intensity_saturation
T2Flair	549	543	6	98.90710	1.0928962	qcfail_background_noise
T2Flair	549	539	10	98.17851	1.8214936	qcfail_motion
T2Flair	549	547	2	99.63570	0.3642987	qcfail_other

DT results

DTI QC summary
modality	Total	Successes	Failures	Percent_Success	Percent_Failure	failcat
DTIb0	1204	1110	94	92.19269	7.8073090	qchuman_DTI
DTIdwi	1340	1224	116	91.34328	8.6567164	qchuman_DTI
DTIb0	1204	1188	16	98.67110	1.3289037	qcfail_orientation
DTIdwi	1340	1324	16	98.80597	1.1940299	qcfail_orientation
DTIb0	1204	1198	6	99.50166	0.4983389	qcfail_phantom
DTIdwi	1340	1334	6	99.55224	0.4477612	qcfail_phantom
DTIb0	1204	1204	0	100.00000	0.0000000	qcfail_intensity_saturation
DTIdwi	1340	1336	4	99.70149	0.2985075	qcfail_intensity_saturation
DTIb0	1204	1154	50	95.84718	4.1528239	qcfail_background_noise
DTIdwi	1340	1286	54	95.97015	4.0298507	qcfail_background_noise
DTIb0	1204	1150	54	95.51495	4.4850498	qcfail_motion
DTIdwi	1340	1270	70	94.77612	5.2238806	qcfail_motion
DTIb0	1204	1190	14	98.83721	1.1627907	qcfail_signal_dropout
DTIdwi	1340	1318	22	98.35821	1.6417910	qcfail_signal_dropout
DTIb0	1204	1188	16	98.67110	1.3289037	qcfail_other
DTIdwi	1340	1324	16	98.80597	1.1940299	qcfail_other

rsfMRI results

rsfMRI QC summary
modality	Total	Successes	Failures	Percent_Success	Percent_Failure	failcat
rsfMRI	2969	2512	457	84.60761	15.3923880	qchuman_rsfMRI
rsfMRI_LR	187	168	19	89.83957	10.1604278	qchuman_rsfMRI
rsfMRI_RL	854	758	96	88.75878	11.2412178	qchuman_rsfMRI
rsfMRI	2969	2960	9	99.69687	0.3031324	qcfail_orientation
rsfMRI_LR	187	186	1	99.46524	0.5347594	qcfail_orientation
rsfMRI_RL	854	838	16	98.12646	1.8735363	qcfail_orientation
rsfMRI	2969	2955	14	99.52846	0.4715392	qcfail_phantom
rsfMRI_LR	187	187	0	100.00000	0.0000000	qcfail_phantom
rsfMRI_RL	854	854	0	100.00000	0.0000000	qcfail_phantom
rsfMRI	2969	2955	14	99.52846	0.4715392	qcfail_spacing
rsfMRI_LR	187	187	0	100.00000	0.0000000	qcfail_spacing
rsfMRI_RL	854	854	0	100.00000	0.0000000	qcfail_spacing
rsfMRI	2969	2949	20	99.32637	0.6736275	qcfail_intensity_saturation
rsfMRI_LR	187	177	10	94.65241	5.3475936	qcfail_intensity_saturation
rsfMRI_RL	854	822	32	96.25293	3.7470726	qcfail_intensity_saturation
rsfMRI	2969	2388	581	80.43112	19.5688784	qcfail_motion
rsfMRI_LR	187	187	0	100.00000	0.0000000	qcfail_motion
rsfMRI_RL	854	854	0	100.00000	0.0000000	qcfail_motion
rsfMRI	2969	2804	165	94.44257	5.5574267	qcfail_signal_dropout
rsfMRI_LR	187	172	15	91.97861	8.0213904	qcfail_signal_dropout
rsfMRI_RL	854	806	48	94.37939	5.6206089	qcfail_signal_dropout
rsfMRI	2969	2968	1	99.96632	0.0336814	qcfail_temporal_noise
rsfMRI_LR	187	186	1	99.46524	0.5347594	qcfail_temporal_noise
rsfMRI_RL	854	846	8	99.06323	0.9367681	qcfail_temporal_noise
rsfMRI	2969	2957	12	99.59582	0.4041765	qcfail_insufficient_volumes
rsfMRI_LR	187	187	0	100.00000	0.0000000	qcfail_insufficient_volumes
rsfMRI_RL	854	854	0	100.00000	0.0000000	qcfail_insufficient_volumes
rsfMRI	2969	2968	1	99.96632	0.0336814	qcfail_FOV
rsfMRI_LR	187	187	0	100.00000	0.0000000	qcfail_FOV
rsfMRI_RL	854	854	0	100.00000	0.0000000	qcfail_FOV

NM results

NM QC summary
NM.QC.cat	Total	Successes	Failures	Percent_Success	Percent_Failure	failcat
overall	478	386	92	80.75314	19.246862	NM_QC_Ratings_Failures
z.coord	478	391	87	81.79916	18.200837	NM_QC_Ratings_Z
sn.vol	478	471	7	98.53556	1.464435	NM_QC_Ratings_SNVol
avg.int	478	473	5	98.95397	1.046025	NM_QC_Ratings_AvgIntensity
sd.int	478	473	5	98.95397	1.046025	NM_QC_Ratings_SDIntensity
max.int	478	473	5	98.95397	1.046025	NM_QC_Ratings_MaxIntensity

do a check based on rsfmri motion – if the qc data “makes sense”

do a check based on DTI – if the qc data “makes sense”

do a check based on NM – if the qc data “makes sense”

do a check based on FLAIR – if the qc data “makes sense”

do a check based on T1w – if the qc data “makes sense”

2. Detailed curation results provide a unique resource in itself

for algorithm developers

we ourselves benefit from this (will show examples)

for scientists

to have confidence (or at least context regarding quality) in looking at automated IDPs derived from these data

3. Processing improves quality/usability further

effect of processing by modality

for each, summarize how processing improves the data
- rsfMRI is “easy” to see - throw away motion corrupted data
- developed methodology specific for PPMI that was informed by this process
- automated QC for these modalities for future work

we provide scientists handles on this so that can be “strict” or “loose” in data selection

we document these parameters

4. Enables investigation of the “meaning” of M3RI at the individual level

5. Empowers investigation of the “meaning” of M3RI at the population level

simlr

## [1] "     <0> BUILD-V <0> BUILD-V <0> BUILD-V <0> BUILD-V <0>    "
## [1] "initialDataTerm: -0.375449406559412  <o> mixer: pca  <o> E:  cca"
## [1] "Iteration: 1 bestEv: -0.0317598345806564 bestIt: 1 CE: -0.0317598345806564"
## [1] "Iteration: 2 bestEv: -0.127153827109486 bestIt: 2 CE: -0.127153827109486"
## [1] "Iteration: 3 bestEv: -0.159247716566849 bestIt: 3 CE: -0.159247716566849"
## [1] "Iteration: 4 bestEv: -0.169491210153665 bestIt: 4 CE: -0.169491210153665"
## [1] "Iteration: 5 bestEv: -0.174768298475065 bestIt: 5 CE: -0.174768298475065"
## [1] "Iteration: 6 bestEv: -0.174768298475065 bestIt: 5 CE: -0.152803217187931"
## [1] "Iteration: 7 bestEv: -0.199710965219991 bestIt: 7 CE: -0.199710965219991"
## [1] "Iteration: 8 bestEv: -0.199710965219991 bestIt: 7 CE: -0.19579644862074"
## [1] "Iteration: 9 bestEv: -0.199710965219991 bestIt: 7 CE: -0.19579644862074"
## [1] "Iteration: 10 bestEv: -0.199710965219991 bestIt: 7 CE: -0.19579644862074"
## [1] "Iteration: 11 bestEv: -0.199710965219991 bestIt: 7 CE: -0.19579644862074"
## [1] "Iteration: 12 bestEv: -0.199710965219991 bestIt: 7 CE: -0.196780449833828"
## [1] "Iteration: 13 bestEv: -0.208143705672403 bestIt: 13 CE: -0.208143705672403"
## [1] "Iteration: 14 bestEv: -0.208143705672403 bestIt: 13 CE: -0.205026650021558"
## [1] "Iteration: 15 bestEv: -0.239617756820512 bestIt: 15 CE: -0.239617756820512"
## [1] "Iteration: 16 bestEv: -0.239617756820512 bestIt: 15 CE: -0.239617756820512"
## [1] "Iteration: 17 bestEv: -0.239617756820512 bestIt: 15 CE: -0.239617756820512"
## [1] "Iteration: 18 bestEv: -0.239617756820512 bestIt: 15 CE: -0.239617756820512"
## [1] "Iteration: 19 bestEv: -0.239617756820512 bestIt: 15 CE: -0.239617756820512"
## [1] "Iteration: 20 bestEv: -0.244185533257034 bestIt: 20 CE: -0.244185533257034"
## [1] "Iteration: 21 bestEv: -0.244185533264272 bestIt: 21 CE: -0.244185533264272"
## [1] "Iteration: 22 bestEv: -0.244185533264272 bestIt: 21 CE: -0.244185533264272"
## [1] "Iteration: 23 bestEv: -0.244185533264272 bestIt: 21 CE: -0.244185533264272"
## [1] "Iteration: 24 bestEv: -0.244185533264272 bestIt: 21 CE: -0.244185533264272"
## [1] "Iteration: 25 bestEv: -0.244185533264272 bestIt: 21 CE: -0.244185533264272"
## [1] "Iteration: 26 bestEv: -0.244185533264272 bestIt: 21 CE: -0.244185533264272"
## [1] "Iteration: 27 bestEv: -0.244185533264272 bestIt: 21 CE: -0.244185533264272"
## [1] 0.06308406
## [1] 0.03358377
## [1] 0.02338849
## [1] "1 0.000182813961727995"

## [1] 0.03914674
## [1] 0.0688519
## [1] 0.02239701
## [1] "1 0.253184809658498"

## [1] 0.04997151
## [1] 0.06109953
## [1] 0.01932642
## [1] "1 0.117369987110731"

## [1] 0.0155129
## [1] 0.09418174
## [1] 0.03088837
## [1] "1 0.0772706971826326"

## [1] 0.0116334
## [1] 0.07219495
## [1] 0.02426768
## [1] "1 0.0245371892594724"

## [1] 0.02914037
## [1] 0.07281831
## [1] 0.02122702
## [1] "1 0.00413298641191162"

## [1] 0.04614809
## [1] 0.04694281
## [1] 0.05216954
## [1] "1 0.444470145275164"

## [1] 0.06048338
## [1] 0.0978308
## [1] 0.01358586
## [1] "1 0.116907067117507"

## [1] 0.07213305
## [1] 0.0697766
## [1] 0.0292922
## [1] "1 0.0735499924260931"

## [1] 0.05090588
## [1] 0.1141006
## [1] 0.01063304
## [1] "1 0.168098675956571"

## [1] 0.02541644
## [1] 0.09387618
## [1] 0.03808487
## [1] "1 0.201933845040915"

## [1] 0.05573603
## [1] 0.09141141
## [1] 0.04255581
## [1] "1 0.01919865559824"

## [1] 0.04590256
## [1] 0.06853646
## [1] 0.02107676
## [1] "1 0.42165835391733"

## [1] 0.05111144
## [1] 0.04577096
## [1] 0.01435047
## [1] "1 0.0708125993718408"

## [1] 0.05406733
## [1] 0.05454088
## [1] 0.01260812
## [1] "1 0.212117989047716"

## [1] 0.02707338
## [1] 0.06951353
## [1] 0.03713177
## [1] "1 0.838761144764153"

## [1] 0.0463601
## [1] 0.01240573
## [1] 0.01356592
## [1] "1 0.147990214086151"

## [1] 0.03481475
## [1] 0.1041289
## [1] 0.02005062
## [1] "1 0.0918154379513776"

## [1] 0.02238462
## [1] 0.05018068
## [1] 0.04757634
## [1] "1 0.0841661059386939"

## [1] 0.03900399
## [1] 0.05628325
## [1] 0.03711137
## [1] "1 0.330606061335976"

## [1] 0.04638839
## [1] 0.04219776
## [1] 0.02991352
## [1] "1 0.180275099839925"

SiMLR permutation summary
permstate	anv	pvalt1	pvalrs	pvaldt
original	0.0001828	0.0024539	0.0869260	0.0035658
perm	0.2531848	0.1140175	0.1861437	0.7066202
perm	0.1173700	0.1843373	0.3435084	0.0409037
perm	0.0772707	0.6311595	0.5723706	0.0103567
perm	0.0245372	0.1506731	0.1488727	0.0217073
perm	0.0041330	0.6336158	0.1130059	0.0017395
perm	0.4444701	0.2083604	0.3016553	0.8922591
perm	0.1169071	0.0584459	0.2420679	0.1599787
perm	0.0735500	0.4102780	0.9401177	0.0123246
perm	0.1680987	0.9676549	0.6917760	0.0314073
perm	0.2019338	0.1558100	0.5523330	0.1498735
perm	0.0191987	0.7096165	0.8340096	0.0017361
perm	0.4216584	0.3479091	0.1755137	0.9776325
perm	0.0708126	0.6606754	0.7714953	0.0112521
perm	0.2121180	0.8806537	0.7828186	0.0378230
perm	0.8387611	0.7557867	0.7384605	0.4004283
perm	0.1479902	0.6723744	0.2996158	0.0424704
perm	0.0918154	0.9887208	0.6955649	0.0145325
perm	0.0841661	0.1468834	0.5218746	0.0270130
perm	0.3306061	0.1082332	0.4396962	0.8688471
perm	0.1802751	0.1296333	0.4942471	0.1411465

## Warning: Using `bins = 30` by default. Pick better value with the argument
## `bins`.

## Warning: Groups with fewer than two data points have been dropped.

## Warning in max(ids, na.rm = TRUE): no non-missing arguments to max; returning
## -Inf

PPMI500 data curation effort

Brian B. Avants^1,2

Lino Becerra¹

Leon Fonville^2,†

Olivia Hampton^1,†

Alexandra Reardon^1,†

Andrew Stenger^1,†

Xue Wang^1,†

Roger Gunn

Overview

the PPMI500 cohort

1. Data quality is overall high

How do we define M3RI quality?

what is the QC protocol?

what are the QC results?

Tables of Manual QC results

T1w results

Flair results

DT results

rsfMRI results

NM results

do a check based on rsfmri motion – if the qc data “makes sense”

do a check based on DTI – if the qc data “makes sense”

do a check based on NM – if the qc data “makes sense”

do a check based on FLAIR – if the qc data “makes sense”

do a check based on T1w – if the qc data “makes sense”

2. Detailed curation results provide a unique resource in itself

for algorithm developers

for scientists

3. Processing improves quality/usability further

effect of processing by modality

we provide scientists handles on this so that can be “strict” or “loose” in data selection

4. Enables investigation of the “meaning” of M3RI at the individual level

5. Empowers investigation of the “meaning” of M3RI at the population level

simlr

PPMI500 data curation effort

Brian B. Avants1,2

Lino Becerra1

Leon Fonville2,†

Olivia Hampton1,†

Alexandra Reardon1,†

Andrew Stenger1,†

Xue Wang1,†

Roger Gunn

Overview

the PPMI500 cohort

1. Data quality is overall high

How do we define M3RI quality?

what is the QC protocol?

what are the QC results?

Tables of Manual QC results

T1w results

Flair results

DT results

rsfMRI results

NM results

do a check based on rsfmri motion – if the qc data “makes sense”

do a check based on DTI – if the qc data “makes sense”

do a check based on NM – if the qc data “makes sense”

do a check based on FLAIR – if the qc data “makes sense”

do a check based on T1w – if the qc data “makes sense”

2. Detailed curation results provide a unique resource in itself

for algorithm developers

for scientists

3. Processing improves quality/usability further

effect of processing by modality

we provide scientists handles on this so that can be “strict” or “loose” in data selection

4. Enables investigation of the “meaning” of M3RI at the individual level

5. Empowers investigation of the “meaning” of M3RI at the population level

simlr

Brian B. Avants^1,2

Lino Becerra¹

Leon Fonville^2,†

Olivia Hampton^1,†

Alexandra Reardon^1,†

Andrew Stenger^1,†

Xue Wang^1,†